Measurement equivalence and latent mean differences of personality scores across different media and proctoring administration conditions

Article history: Keywords: Measurement equivalence Latent means Personality Paper-and-pencil test Computer-based test Proctoring a b s t r a c t Despite substantial interest and research in measuring personality, little is known about the measurement equivalence and mean differences in scores on personality measures across different administration conditions. The aim of the present study was to assess measurement equivalence and latent and observed mean differences of scores on the Big Five factor markers from the International Personality Item Pool across three conditions: paper-and-pencil proctored, computer-based proctored, and computer-based non-proctored. Undergraduate students (N = 401) from a Midwestern university responded to the personality questionnaire in one of the three conditions. Results indicated configural, metric, scalar, and invariant uniqueness equivalence for four of the five scales across the three conditions; Conscientious-ness scores showed partial metric equivalence across computer-based proctored and computer-based non-proctored conditions. Apart from latent and observed mean differences for Emotional Stability scores in paper-and-pencil proctored vs. computer-based non-proctored conditions, no significant differences were found for the other four personality scales. These findings justify both collection and comparison of personality data using the Big Five factor markers and similar personality assessments across the three conditions. Future research should attempt to replicate the findings of the current study in high-stakes environments. Since the popularization of computers and the emergence of Internet, more and more organizations have begun to switch from traditional paper-and-pencil (PP) formats of their psychological measures to computer-based (CB) formats due to the many advantages of CB testing (Stanton, 1998). CB testing, in turn, has led to the emergence of non-proctored testing where a respondent completes a measure in the absence of a proctor (Tippins et al., 2006). In fact, many organizations have developed CB versions of their measures for human resource practices (Mead & Blitz, 2003; Salgado & Moscoso, 2003) and educational purposes (Wise & Plake, 1990). This interest in CB testing hinges on the advantages that the CB administration offers relative to the traditional PP format. For example, using computers and Internet for collecting psychological data usually requires fewer resources for test administration, maintenance, and updates, and at the same time can ensure more efficient and accurate data entry and scoring (Booth-Kewley, Larson, & Miyoshi, 2007). The Internet can also provide an easier access to large test-taker pools, which is difficult to achieve with PP methodology (Buchanan & Smith, 1999). However, the widespread use of technology for psychological …